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An alternative to monitoring instruction via standardized tests is 
proposed for objective-based individualized instructional programs, The 
procedures can be applied to data from curriculum-embedded assessments, 
and are s traightforward to use whc-n the data is in matrix or keysort 
form, Additionally the procedures are computerized, An application in 
reading is given, and the diagnostics made from the tabular and graphic 
summaries of the data are illustrated. When used in conjunction with the 
setting of performance goals by the staff, the monitoring procedures 
assist the staff in focusing upon school-vide priorities and the feedback 
provided annually or at intervals through the school year is used to im- 
prove individualized instruction. 
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MONITORING THE PROGRESS OF TIIE GROUP IN AN INDIVIDUALIZED 
READING PROGRAM BASED ON BE NAVJ. ORAL OBJECTIVES 



Monitoring group progress has traditionally been accomplished through 
the administration of standardized achievement tests. When the instruc- 
tional program is based on behavioral objectives and each child progresses 
at a rate appropriate for him, the comparison of the mean of the group 
with established norms does not provide adequate information for assess- 
ing attainment of the specific objectives of the program. Another short- 
coming is the lack of information useful in assessing differentiation in 
instruction for pupils of various characteristics, as indicated by varia- 
tions in their achievement. Few systems for monitoring the progress of 
the group when the instructional program is based upon behavioral objec- 
tives have been proposed to replace or supplement standardized tests. One 
possible system that would utilize standardized test data to better advan- 
tage entails the analysis of specific content subscalcs which relate to the 
particular curriculum objectives. However, the content in such subscales 
often must be forced to "fit" the program objectives. The purpose of this 
presentation is to explain an alternative monitoring system inherent in a 
behavioral objective-based program and to illustrate diagnostics that can 
be found in it. 

The set of behavioral objectives upon which the illustration is based 
Is taken from the Word Attack component of the Wisconsin Design for Reading 
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Skill Development (the Design) , a product of the Wisconsin Research and 
Development Canter for Cognitive Learning. In this program, 45 objectives 
are arranged into four sets or levels (A through D) of progressive diffi- 
culty. It is presumed that upon attainment of all 45 discrete objectives 
the student will be able to decode independently structurally and phoni- \ 
cally regular v/ords and will knew on sight common irregular words. The 
attainment of this terminal goal is anticipated as early as the end of 
the 4th year in school (Grade 3), or at some later point in the intermediate 
years. It the pupil is to attain the terminal goal at the expected time, 
it is necessary that he make regular progress during the primary and early 
intermediate years. 

The procedure developed links attainment of the terminal goal 
with attainment of the discrete objectives and serves to focus staff effort 
on an educational priority by providing feedback on the progress of groups 
of pupils tov/ard the terminal goal. Tie term "group" as used here will 
not refer to class-size groups but to all children at a particular age- 
grade level. Specifically, the procedures involve setting interim 
performance goals to be assessed on an annual basis. These inter- 
im goals are simply a target in terms of the average percent of 
skills to be mastered by children of similar characteristics. The charac- 
teristics used to stratify children for this analysis (not to group for 
instruction) may include IQ and/or reading achievement , Other information 
which is at hand and relevant, including teacher judgment, may serve to 
subdivide the pupils. In setting the target percentages for each subgroup, 
a school staff may at the outset be guided by the intentions of the program 
developers relative to the typical age-grade placement of particular skills. 
In the case of the Design, the development staff has indicated that Level A 
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corresponds to the content ordinarily introduced in Kindergarten, Level B 
in Grade 1, etc. The performance goals should also take into account 
the past performance of children in the school and reflect neighborhood 
and other environmental circumstances which affect the academic progress 
of individuals in the school. ^ 

A new school which adopts the Wisconsin Design for Reading follows 
this strategy: first, the test baetery is given at the appropriate grade 

levels. The percentages of skills mastered are calculated for subgroups 
of children and compared with the performance goals recommended by the 
developers. See Table 1. Then, after consideration of the limitations 
(or strengths) of its particular group of students, interim goals are 
set for each age-grade group in the school for the ensuing year. At the 
end of each school year progress toward these interim goals is assessed 
and new goals set. In most instances the second set of goals will be some- 
what higher than the first and reflect more closely the developer's projec- 
tions for completion of the program goals. Additionally, there should be 
a closer fit between the goals projected and those attained. A school's 
performance should generally match the developer's target as presented in 
Table 1 after three to five years in the Design if its student population 
has a normal distribution of ability and is unencumbered by home and neigh- 
borhood social and economic problems. Note that the ultimate goals (Table 1) 
are expressed simply in terms of percent of children mastering all skills at 
a level, or "checking out" of the levels of the Design, while the interim 
projections ate expressed as percent of skills mastered by given children 
at a given level. 

In Table 2 the performance goals set in the fall by ono school 
in southern Wisconsin which was using the Design are presented before the 
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slashes* The staff took into account the fact that about half of the 
student population is of lov? socio-economic background. They gave serious 
consideration to each objective in a level and estimated which of the ob- 
jectives groups of children of various characteristics could master. For 
example, there are 13 skills at Level B. Estimating that those children 
of average intelligence could attain on the average 8 of the 13 skills by 
the end of their third year in school (Grade 2), this performance goal 
was set at 62%. The goals were set for the end of the first, third, fifth 
and seventh years since children typically spend two years in a Unit work- 
ing with a team of teachers and other personnel. 

At the end of the school year actual performance data were gathered, 
which appear immediately to the right of the slashes in Table 2. Since 
no children in their first year fell into the lovrest IQ band, we cannot 
evaluate the projection in the first cell. In the other cells the perfor- 
mance data are remarkable for their consistency when looked at either 
within IQ bands across age levels or within age levels across IQ bands. 

In the former case higher age group within a particular IQ band outper- 
formed the preceding age group without exception. In most cases the goals 
set were met and even surpassed; this was probably due in part to the fact 
that the school had used segments of this reading curriculum for two years. 
We would not usually expect so many targets to be surpassed when a school 
is initially implementing the. Design. 

An illustration of the feedback provided by the summary data at the 
end of the year may be found by considering the middle set of figures 




flihose following the slashes) in the first column of Table 2. There is 
little variation in the performance of children in the various IQ bands. 
While this result may reflect the low reliability of the tests for IQ used 
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with this age group, it is also possible that the staff is not implement- 
ing an individualized program sufficiently well. It can also be seen in 
Table 2 that the instructional staff for children in their third year of 
school may have underestimated student capabilities since this group, 
particularly in Levels C and D, consistently surpassed the goals 
set. It was imperative then that these projections be changed signifi- 
cantly for these children so that expectations, particularly for the higher 
IQ groups, were more realistic. The new projections for this school for 
the 1970-71 school year are shown in parentheses and reflect modifications 
based on the 1969-70 results. 

The performance data may aJ so be presented in graphic form. The pro- 
files of the four age groups for the 80-94 IQ band are shown in Figure 1, 
again revealing consistency across age levels. Each higher age group with 
the IQ band outperforms the preceding age group without exception. It is 
also evident that there is a good distribution of achievement and that even 
children of similar characteristics are spread out in attainment of objectives. 
For example, children completing their fifth year have mastered all of 
Level A and mese of Level B skills. From column 2 it may be inferred that 
a few are still working in Level B while most have gone on to work in 
Levels C or D. 

Looking at the data for an age group one sees a similarly consistent 
profile wherein each IQ group 1 s performance is distinct. Figure 2 illus- 
trates the point for those completing the third year of school. Notice, 
however, that there was little difference between the higher two IQ groups. 
This tended to be true at all age levels, perhaps because the school has 
small numbers of pupils in the highest IQ band. The results should draw 
the staff f s attention to the need to plan an instructional program that 
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is differentiated fcr the few above average students. 

Data flora third year pupils in another school are presented in 
Figure 3, and provide an illustration of another diagnostic. The pro- 
file of this group of children in the Level C skills suggests a graded 

approach to the placing of skill development; having three IQ groups spread 

* ./ 

out in Level C skills is comparable to having three groups of children work 
ing at different points in a second grade basal reader program. 

Although a school staff nay require assistance in interpreting such 
graphical representations of the data, the monitoring system itself is 
easily operationalized in the school setting by the teaching staff; 
if attainment of objectives is recorded on individual charts or on keysort 
cards, the calculation of the percentages is straightforward. A program 
is available for analysis through a system of computer management of 
instruction. 

In initiating the system it has been suggested that targets be pro- 
jected before data is collected since most school staffs who examine the 
behavioral objectives with care are able to judge which skills can be mas- 
tered by a particular group of children. However, an alternative approach 
is to collect data at the end of the first yea* of program imp le mentation, 
and to then use it in establishing the goals in subsequent years. A 
school whose staff wishes to improve pupils' performance with respect to a 
particular priority would set higher performance goals the following 
year, whereas one \vhose self-evaJ uation was highly positive night retain 
about the sane performance goals. In most instances some modifies- 
tlons of the performance goals should be expected on the basis of 
the diagnostics provided by the tabular or graphic analysis. 
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Finally, when the research linking the performance goals to other 
more familiar criteria, particularly standardized tests, is complete, per- 
haps the tvro sets of criteria can complement each other. Given a transla- 
tion of performance data gathered from this new procedure to standardized 
test criteria and vice versa, the administrator who monitors instruction 
should have sufficient information co report reading progress of pupils 
in meaningful terms to the community as well as to measure the effectiveness 
of reading instruction in his school. In serving the administrator as well 
as the instructor, curriculum-embedded tests will then yield Information which 
functions in program evaluation as well as individual assessment. 
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Table 1 



Projected Percent Of Children Who Should 

Master All Skills At A Level: 

Possible Performance Goals For A School In Which 
Reading Achievement Is About A Grade Level 

I 



IQ Range 


Skill 

pevelop- 

ment 


Level 


Years 


of School 


Completed 








1 


2 


3 


4 


5 


00-79 


A 


20 


80 


100 


100 


100 




B 


0 


10 


40 


75 


100 




C 






A 5 


25 


75 




D 










25 


80-94 


A 


60 


100 


100 


100 


100 




B 


5 


60 


90 


100 


100 




C 




5 


60 


90 


100 




U 






5 


25 


75 


95-109 


A 


95 


100 


100 


100 


100 




B 


20 


90 


100 


100 


100 




C 




20 


90 


100 


100 




D 






20 


70 


95 


110-140 


A 


100 


100 


100 


100 


100 




B 


50 


100 


100 


100 


100 




C 


5 




100 


100 


100 


o 


D 






50 


95 


100 
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Table 2 

Percent of Word Attack Skills Mastered at Given Levels: 
Projections and Performance , Janesville Wilson School 
1969-70; Projections 1970-71 



H H fl H 
H O CJ 0 
•H > 0 > 
^ 00,0 



IQ 



Years of School Completed 
3 5 



60-79 


A 


42/-* 


(29)*** 


100/91 (93) 


100/10C 


(100) 


100/100 


(100) 




B 


0/-**(0) 


42/26 (34) 


70/75 


(85) 


84/100 


(100) 




C 


01 - 


(0) 


0/0 (6) 


35/28 


(50) 


55/88 


(94) 




D 


01 - 


(0) 


0/0 (0) 


0/6 


(25) 


14/63 


(71) v. 


80-94 


A 


70/54 


(57) 


100/97 (99) 


100/100 


(100) 


100/100 


(100) 




B 


7/2 


(8) 


52/54 (69) 


91/93 


(100) 


100/100 


(100) 




C 


0/0 


(0) 


0/27 (44) 


35/62 


(100) 


70/99 


(100) 




D 


0/0 


(0) 


0/4 (6) 


14/26 


(57) 


56/81 


(100) 


95-109 


A 


84/64 


(88) 


100/100(100) 


100/100 


(100) 


100/100 


(100) 




B 


14/2 


(23) 


63/77 (77) 


100/99 


(100) 


100/100 


(100) 




C 


0/0 


(6) 


21/53 (66) 


80/93 


(100) 


100/100 


(100) 




D 


0/0 


(0) 


0/14 (14) 


70/49 


(95) 


84/93 


(100) 


110-140 


A 


100/73 


(100) 


100/100(100) 


100/100 


(100) 


100/100 


(100) 




B 


21/5 


(38) 


77/81 (96) 


100/100 


(100) 


100/100 


(100) 




C 


0/0 


(ID 


21/59 (91) 


100/97 


(100) 


100/100 


(100) 




D 


0/0 


(0) 


7/16 (43) 


100/69 


(100) 


100/96 


(100) . 



* Projected Percent of Mastery /Actual Performance 
** Indicates no subjects in this category 
*** 1970-71 projections 

o 

ERIC 



i o 



100 



90 

80 

70 

60 

c 50 
£40 

30 

20 

10 

0 




Years of 

School 

Completed 




A B C D 

Skill Development Levels 

Figure 1 Percent of mastery of reading skill for 4 age-grade 
groups whose IQ ranges from 80-94 



& 



ERIC 



13 




Percent 




r* 

E> • 0 

0 0 4 

• 00 



IQ Range 



110-140 




95-109 




80-94 




60-79 



Figure 2 Percent of reading skill mastery for 4 IQ groups 
the end of the third year of school 
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Skill Development Levels 



Figure 3 Percent of Pveading Skill Mastery 
for 3 IQ Groups at the End of the 
Third Year of School 
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